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ABSTRACT 


In  this  paper,  the  author  proposes  two  methods  of  estimation  of  the 
regression  coefficients  when  the  errors  are  not  distributed  identically 
and  independently  and  are  of  nonzero  mean.  The  estimates  proved  in  this 
paper  are  shown  to  be  strongly  consistent  and  mean  square  consistent. 
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1 .  INTRODUCTION 


Suppose  we  have  a  system  of  linear  equations 


Ti  ■  Y1  + 


•  *  X1  6p.  1 

P 


1,  2,  m 


CD 


where  Y. ,  X.  ,  i  =  1,  2,  ....  n,  j  *  1,  2,  ...»  p  are  known  while 
J 

Bl’  **•*  Bp  are  t^ie  un*(nowns •  There  are  many  ways  to  define  a  solution 

for  the  determination  of  S'  =  ( S -j »  ...»  8p).  One  well  known  way  is 

the  so-called  Chebyshev  approximation,  which  seeks  to  minimize 

■  max  |Y,  -  XI  | 

-  l±i±n  ~ 

where  X.  =  (X.  ,  ....  X.  )'.  Such  a  solution,  denoted  by  B.  can  be 

H  p 

computed  by  the  method  of  linear  programing  (see  [2]  and  [6]). 

Now  if  Y.,  Y£.  ...  are  observed  with  random  errors,  then,  instead 
of  (1),  we  have  the  linear  regression  model 


Y.  *  X'.B  +  e. ,  i  =1,2,  ....  n  (2) 

I  I  I 

where  e^,  .. .,  en  are  random  errors.  Usually,  in  (2),  it  is  assumed  that 
the  expectations  of  errors  are  zero  and  have  finite  second  moment  with 
orthogonality  or  asymptotic  independence.  To  solve  this  problem,  one 
can  use  the  famous  least  square  estimation  (LSE)  method.  In  the  literature, 
there  are  a  lot  of  papers  concerning  with  LSE  and  many  important  results 
are  obtained  (see  [1],  [3],  [5]).  But  the  unbiasedness  and  consistency 
of  LSE  strongly  depend  on  the  assumption  that  the  expectations  of  errors 
are  zero,  and  this  assumption  is  not  realistic  sometimes.  The  means  of 
errors  of  measurements  may  be  different.  Similarly,  it  is  not  always 
realistic  to  assume  that  the  errors  are  distributed  independently.  In 
such  situations,  it  is  of  interest  to  obtain  consistent  estimates  of  B. 

In  this  paper,  we  propose  two  methods  of  obtaining  consistent  estimates 
of  S. 


Both  of  these  methods  are  motivated  by  the  so-called  Chebshev  approximation. 

The  first  method  is  to  use  the  measure 

Q  (3)  =  max  (Y.  -  X'.S)  -  min  tY.  -  X*8) 
n  -  l<i<n  1  l<i<n  1 

A 

which  is  never  negative.  So  we  can  find  the  solution  Sn  which  minimizes 

A 

Qn(8).  We  refer  to  B  as  MD  estimate  Cthat  is,  the  estimate  based  on 

Maximum  Difference  between  residuals). 

The  second  method  is  to  use  the  measure 

Qn(B)  =  max  |  Y .  -  X'.s|. 

*  l<1<n  ~  ~ 

Denote  by  s’  the  value  of  S  which  minimizes  "Q  (B).  We  shall  call  "s 
-n  „  n  _  .n 

MA  estimator  (that  is,  the  estimator  based  on  the  Maximum  Absolute  value 
of  residuals) . 

A 

In  Section  2,  we  shall  prove  the  strong  consistency  of  6  whereas 

•»  I  * 

in  Section  3,  we  shall  prove  its  mean  square  consistency.  In  Section  4, 
we  will  prove  that  estimates  of  endpoints  of  error  support  based  on  the 

A 

residuals,  with  respect  to  8n,  are  strong  by  consistent,  and  establish 
the  strong  consistency  of  the  estimate  of  second  moment  of  errors,  based 

A 

on  sn»  when  the  error  sequence  is  stationary  and  ergodic.  In  Section  5, 
we  establish  the  strong  consistency  of  s’  and  its  mean  square  consistency 
is  given  in  Section  6.  In  Section  7,  we  shall  prove  the  consistency  of 
estimates  of  the  largest  value  of  endpoints  and  that  of  the  second  moment 
of  errors  when  the  error  sequence  is  stationary  and  ergodic.  Here  we 
emphasize  the  fact  that  we  do  not  use  independence  (even  in  the  asymptotic 
case)  of  the  error  sequence  in  proving  our  results. 

For  example,  let  n  =  1,  2,  ...  be  i.i.d.  random  variables  with 
common  uniform  distribution  over  the  interval  [0,1].  Define  the  errors  as 


’T’VTTV 


i^r ' -.innM 


e2k+i  =  ^  =  ^  *  2*  •••»  1  =  2 *  ****  2  * 

Such  an  error  sequence  satisfies  all  conditions  in  our  theorems  but  it 

is  not  asymptotically  independent. 

Of  course,  if  the  errors  have  a  common  mean,  our  model  will  turn  out 
to  be  the  usual  linear  regression  model.  In  this  paper,  we  do  not  assume 
the  errors  have  common  mean.  For  example-,  let  en  be  independent  r.v. 
distributed  on  (0,1)  and  with  density  pn(x)  e  >  0.  Also,  assume  that 
E(en)  are  not  equal.  Define 

e2k+i  =  “k  ei  +  ek  ’  1  =  1  *  2 . 2  »  k  =  1 ,  2 ,  ...  . 

In  this  case,  LS  method  does  not  work  but  our  method  still  works. 
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2.  STRONG  CONSISTENCY  OF  B 

-  n 

In  this  section  we  shall  prove  the  following  theorem: 

Theorem  1 .  Suppose  that  e.  has  a  support  included  in  the  bounded  in¬ 
terval  [  a-j »  ,  and  for  any  subsequence  (e^  :  j  =  1,  2,  ...}  of  (e^) 

J 

we  have 


lim  ei  =  a2»  lim  e^  =  a]  »  a*s*  (3) 

Also,  suppose  that  {X.}  is  bounded,  and  for  any  non2ero  p-vector  d, 

lim  x'd  does  not  exist.  Then 
n-**>  -n~ 

A 

lim  B  =  6 ,  a.s.  (4) 

n-x=  *’ 

Proof.  Several  preliminary  facts  are  in  order. 

1.  Define  a  function 

g(a )  =  lim  ot'x„  -  lim  o' x 
,  -  ,n  - -  ..n 

on  R*5,  then  g(a)  is  continuous  on  r'3,  and  g(a)  >  0  when  a  ^  0. 

The  second  conclusion  follows  directly  from  the  assumption  of  the 
theorem,  and  the  first  is  an  easy  consequence  of  the  boundedness  of  (xi }. 

Denote  by  B  the  surface  of  the  unit  sphere  in  R^  i.e.  B  =  {ct:  ||a||  =  1}. 
Because  B  is  compact,  g  is  continuous,  and  g(a)  >  0  for  a  f  0,  we  have 

A 

c=infg(a)>0.  (5) 

aeB 

2.  Define  B  and  c  as  above.  Then  we  can  find  a  positive  integer  m 
and  a  subset  D  =  (a^ ,  ....  a^}  of  B  such  that  for  any  a  s  B,  there  exists 
a.  €  D  with  the  property 


The  proof  is  obvious.  If  H^H  <  M,  j  *  1,  2,  ....  then  we  need  only 
to  find  m  large  enough  so  that  there  exists  a  ^  -  net  of  B  consisting  of 

m  points.  This  ^  -  net  is  chosen  as  D • 

For  every  e  D,  we  can  find  out  two  subsequences  {x.,  .}“  and 
-1  j=l 

(x.0,r  of  {x  }  sych  that 
-,2j  j =1  -n 

lim  alXiii  =  1  i m  a!xn,  lim  alx^.-  =  linn  alxn. 
j-*»  J  n-*»  j-*»  J  n-*» 

Denote  the  values  of  y  and  e  corresponding  to  x.  .  by  y.  . ,  e .  . ,  r  =  1,  2 

~i  rj  irj  n  r  j 

1-e-  1f  *1rj  =  2k-  thenyirj  yk*  eirj  =  V  Put 


Eu  =  (lim  eUJ  =  a2>,  E,2 


(lim  e  =  aj) 

1-X» 


E  '  .n=1(Eun  Ei2>- 


3.  P(E)  =  1. 

In  fact,  since  PCE.jj)  =  P(E-2)  =  1*  i  =  1.  ...»  m  by  (3),  we  get  the 
result. 

Employ  the  symbol  w  to  represent  a  real  number  sequence  (c-j,  c^,  ...) 
al  1  C1  1  a2»  i  =  1»  2,  ...  .  The  event  E  determined  by  (7)  can  be  viewed 
as  a  subset  E  of  the  w-space  i.e.  E  =  { (e -j ,  e2>  ...)  e  E}. 

4.  For  any  €  >  0,  t  >  ®,  a.  e  D,  put  d  =  B  +  to.,  then  for  any 

1  W  t  *«*  •»  1 

a)  e  E,  there  exists  N^(oj)  such  that 


max  (y,.U)  -  x.’d. )  -  min  (y.U)  -  x'd.)  >  a0  -  a.  +  4tc 


when  n  _>  N.  (cu) . 

The  proof  is  as  follows:  For  simplicity  of  notation  we  shall  use 
yk,  ek  to  express  ykU)  and  e^ ( to) -  Then 

\  -  ,*Ldt  ’  -xk-s  +  ek  -  -xk(5  +  l?i>  =  ek  -  “kV 


Take  n  >  0  whose  value  will  be  given  later.  Since  Tfm  e . , .  =  a2  and 

j-Vo.  J 

lim  a-x. .  .  =  lim  a-x  ,  we  can  find  j.  =  j,(n,w)  so  that 

•  -1-1  I  J  ‘77  •  1  «*'•  »  1 


611j1  -  a2  "  n 


(1 


a!x.,  .  <  lira  a'.x  +  n.  0 

Take  =  N^U.w)  large  enough  so  that  when  n  >_  we  have 

-il j1  €  {?1»  -n}* 

Then  by  (9)-(ll),  when  n  >_  we  have 


max  (y.  -  x'd  )  j>  e.,  .  -  tx. , .  'a.  >  a2  -  n  -  t(T1m  alx  +  n) .  (1 

l<k<n  k  '■k~t  1IJ1  -1  -1  c  ih-  '1"n 


Similarly,  we  can  prove  that  there  exists  N-2  =  N.2(n,w)  such  that  when 
n  >  Ni^  we  have 

min  (y.  -  x'd  )  <_  a.  +  n  -  t(TFm  a!x„  -  n).  (' 

l<k<n  K  ~KrZ  1  n^>  -1-n 


Put  N.  =  N .  (n  ,oj)  *  max{N.  ^  (n»oj)j  N.^(n,oj)}.  Then  using  (12),  (13)  and 
noticing  the  definition  of  c  in  (5),  we  get 
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max  (y.  -  x'd  )  -  min  (y.  -  x'd  ) 
l<k<n  K  l<k<n  K 

>_  -  n  -  t( lim  alx  +  n)  -  (a.j  +  n  -  t(lim  a-x^  -  n) 

n-x°  ~  "  n-*» 

>_  a2  -  a.|  -  2(t  +  1  )n  +  tc.  (14) 

Now  specify  n  =  Since  tvfj  1  g'(J+ry  f°r  t  >  s,  it  follows 

from  (14)  that  for  t  _>  e 

max  (y.  -  x'd  )  -  min  (y.  -  x'd  ) 

1  <k<n  K  "  K“ 1  1  <k<n  K  * K” 1 

>  a2  -  a]  -  2(t  +  1 )  57 1 +TT  +  tc  =  a2  "  al  +  "3tC” 

This  proves  the  assertion. 

*» 

Now  turn  to  the  proof  of  theorem  1.  Fix  cu  «  E  and  e  >  0,  and  again 

use  y. ,  e.,  etc.  to  express  y^U),  e^U),  N..  (u>) .  Put  N  =  N(u>)  = 

max  N^  (oj ) .  Suppose  that  l  e  RP  satisfies  \\Z  -  s|J  >_  e,  then  l  can  be 
1  _<i  <m  1  ~  “ 

written  as  l  =  B  +  ta,  t  >_  s,  a  e  B.  By  2,  we  can  choose  a.  «  D  so  that 
(6)  is  true.  Remember  that  dt  =  S  +  ta^ ,  we  have 

yk  -  -xk-4  3  yk  ■  .xk(f + 1?>  -  yk  -  .xk(.dt + t(?  -  ?t>> 

■  yk  •  ,xLdt  -  ‘ill1;  -  i  yk  -  ?k?t  -  tc/4 

and 

y k  -  xj^f  <  yk  -  x^dt  +  tc/4. 

From  these  inequalities  and  (8),  we  have 


for  n  _>  N.  On  the  other  hand,  we  know  that 

max  (y.  -  x'g)  -  min  (y.  -  x's) 
l<k<n  K  1<k<n  K 

=•  max  e.  -  min  e.<  a,  -  a,  . 
l<k<n  K  l<k<n  K  c  1 

So  we  have  Qn(£)  >  Q  ( 3 )  for  n  _>  N  and  J|£  -  s||  >_  s.  According  to 

*  A 

the  definition  of  S  ,  we  have  [[ 3  -  sf  <  e  when  n  >  N.  Since  e>  0  is 

*  >  a* 

arbitrary,  this  proves  that  lim  8  =  S  when  oi  €  E.  The  proof  of  Theorem  1 

n-H» 

is  concluded  in  view  of  3. 

Let  us  mention  two  important  examples  of  this  theorem, 

1.  {e  - }  is  a  m-dependent  and  identically  distributed  sequence  and 
the  support  of  e-  is  bounded,  where  m  is  a  positive  integer. 

Let  a-j,  express  the  infimum  and  supremum  of  the  support  of  e-j 
respecti  vely, 

(I)  m  =  1, 

It  is  obvious  that  1  i m  e.  =  a~,  1  i m  e.  =  a.j  for  any  subsequence 

j-*30  j  j-*30  j 

(ei  }  of  (e . } . 

J  (II)  m  >  1 . 

Take  arbitrarily  a  subsequence  {e.  }  of  { e .  > .  We  can  further  choose 

'j  s  1 _ 

an  iid.  subsequence  {e.}  from  (e.  }.  By  (I )  we  have  lim  e.  =  a9,  a.s. 

1  I  >  •  1  L. 

_  _  J 

So  a.,  _>  lim  e^  >_  lim  e.  =  a~,  a.s.  In  a  similar  way  we  prove  the  other 

j-ym  j  i-x* 

asserti on  of  ( 3) . 


9. 


2.  Let  {e.j }  be  a  stationary  sequence,  the  support  of  e-j  being  bounded 
(a-j,  a2  denote  the  left  and  right  endpoint  of  the  support)  and  suppose 
that  the  following  condition  (weak  dependency)  holds: 

limfsup{|p(A)p(Bn)-p(ABn)  |  :  Aecr(e  J  Bn€o(e.+n,ei+n+1  ,..<.)}]  =0  (15) 

n-*»  ”  1 

for  any  fixed  i,  where  a(e. ,  e.,  ...)  express  the  a-field  generated  by 

*  vl 

6«j  6  j  4  •••  • 

For  this  case  we  again  take  an  arbitrary  subsequence  (e.  }  of  {e..}, 

3 

for  fixed  e  >  0,  put  E-  =  {e.  <_  a?  -  It  needs  only  to  verify  that 

J  J 

P(Ev  Ev+1  Ev+2  =  °»  for  any  v*  (16) 

Use  Cin  to  denote  the  quantity  under  the  limit  sign  of  (15).  Fix  r  >  v  and 

take  n^  large  enough  to  make  C.  n  <  l/2r.  Notice  that  v +  ni  iw 

r  1  r+ 1  ^ 

Further,  choose  n9  large  enough  to  satisfy  C.  <1/2  .In  general, 

6  1 r+n. n2 


after  determining  n  .  we  choose  n  large  enougn  so  that  Cj  „  . 

r+n,+. .  ,+n  m+1 

,  ,„r+m  T.  i  m 

<1/2  .  Then 

P(EV  ••») 

-  P^Er  Er+n^  Er+n1+n2+n3  ***^ 

—  1/2  +  P^Er+n1  Er+n1+n2  Er+n1+n2+n3  ,00^Er^ 

<  l/2r  *  l/2r+1  +  P(Er+vr,2  E,^^  ...)P(Er)P(Er+ni) 

*  *  * 

<  (l/2r  ♦  l/2r+'  ♦  ...)  *  P(Er)P(E  )...P{E  ) 

1  l  m 

for  any  nature  number  m.  Since  P(Er)  =  P(Er+^)  =  ...  <  1,  letting  m 
in  the  above  expression  we  get 


3.  MEAN  SQUARE  CONSISTENCY  OF  |n 
In  this  section  we  shall  prove  the  following  theorem: 


Theorem  2.  Under  the  assumption  of  theorem  1  we  have 

lim  E (y 6 n  -  Bl2)  =  0. 
rr*° 

Proof.  Define  h(a)  =  lim  T  max  ct'x..  -  min  ct'x..]*  Because  {x-j}  is 

n-*=°  "l<_i_<n~  ~  1  <n~  “ 

bounded,  h (ot)  is  finite  everywhere  and  further,  it  is  easily  seen  from 
the  assumption  of  theorem  1  that  h(a)  >  0  when  a  t  0,  and  h(a)  is  contin¬ 
uous  everywhere  on  R*5.  So  the  infremum  of  h(a)  on  the  surface  of  unit 
sphere  B  is  greater  than  zero  :  c  =  inf{h(a):  a  e-  B>  >  0.  Find  on  B  a 
finite  subset  D  =  {5-j ,  ...»  Sq>  with  the  following  property.  For  any 
a  e  B,  there  exists  i  e  (1,  ...»  q}  such  that 

sup{ |  (a  -  a.)'xJ:  j=l,  2,  ...}  <  c/4.  (17] 

-  *•  l  **  J 

Now  we  prove  the  existence  of  N  such  that 

max  x'.a  -  min  x'.a  >  c/4,  for  every  a  e  B  when  n  _>  N. 

Ui^n”1”  ld<n*u 

•m 

In  fact,  by  the  definition  of  c,  we  can  find  out  N  such  that 
max  x/a.  -  min  x£a.  >  3c/4,  i  =  1,  ...»  q 

l±k±n  If*!" 

when  n  ^  N.  For  any  a  e  B  choose  a..  such  that  (17)  holds,  then  we  have 

max  x/a  -  min  x/a  >  (  max  x/a.  -  c/4)  -  (  min  x/a.  +  c/4) 
l<k<n~  “  1 <k<n~  "  ~  1 <k<n"  ~  1 <k<n~ 


=  max  x/ a .  -  min  x/a.  -  c/2  >  3c/4  -  c/2  =  c/4 
l<k<n“K  l<k<n”K"1 


when  n  _>  N.  Now  put  t  =  2^  -  aj)/c.  If  Z  s  R*3  anc  ||  £  -  8 1|  >  t,  then  l 
can  be  written  as  £  =  8  +  ta,  a  e  B,  t>  t.  Thus  for  n  ^  N ,  we  get 
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Q  (£)  =  max  (yk  -  x^)  -  min  (yk  -  x^) 

*  l<k<n  K  K  l<k<n  K  K 

*m  m 

-  max  (e,,  -  txja)  -  min  (ek  -  txAa) 
l<k<n  K  '  l<k<n  K 

>  t(  max  x/a  -  min  xka)  -  (  max  -  min  ek) 

l<k<n~K“  l£k_<n~K~  1  <k<n  K  1  <k<n  K 

>  tc/4  -  (a2  -  a-,)  >  2(a2  -  a.,)  >  Qn(s). 

A  A 

This  shows  that  {<3^,  eN+^ ,  ...}  are  uniformly  bounded.  Since 

<*.  * 

P(lim  3  =  3)  =  1  by  theorem  1,  the  assertion  of  theorem  2  follows  from 
n-*» 

the  dominant  convergent  theorem. 


< 
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4.  ESTIMATION  OF  THE  ENDPOINTS  OF  THE 
SUPPORT  AND  THE  SECOND  MOMENT 


Define 


min  (y. 
l<i<n  1 


x.i!n>*  *2n  *  ,**^1  -  x-iln>' 


Theorem  3.  Under  the  assumption  of  theorem  1,  aln,  a2n  are  strong 

A  A 

consistent  estimates  of  a^,  a2  respectively.  (So  a2n  -  a^n  is  a  strong 
consistent  estimate  of  a2  -  a^) 

Proof.  Take  a„  for  instance.  Because  Tim  max  e„.  >  Tim  e„,  by  (3), 

rv*»  l<i<n  1  “  n 

we  have  P(lim  max  e*  =  a9)  *  1.  Now 
n-x=°  l<i<n  1 

max  (y.  -  xje)  -  max  (y,  -  xjs  -  xj(^  -  6)) 
l<^i<n  l<i<n 

=  max  (y.  -  x]$)  +  =  max  e.  +  J 

l<i<n  1  n  i<i<n  1  n 

A  A 

where  |Jn|  <  sup  ||x.||  ||6  -e||..  Because  lim  B_  *  e,  a.s.  in  view  of  theorem 
n  -  -n  ~n 

1  and  (x.)  is  bounded,  we  derive  that  lim  J  »  0,  a.s.  Combining  this  with 

P(lim  max  es  =  a,)  =  1 ,  we  get  lim  a«  »  a«,  a.s. 
ih-  k<i<n  1  c  cn  c 

Now  suppose  that  e-, ,  e2,  ...  is  a  strictly  stationary  ergodic 
sequence.  As  an  estimate  of  Ee^,  we  use  Sn  =  1/n  \  (y..  -  x|Bn)  . 

Theorem  4.  Suppose  that  {e..}  is  a  strictly  stationary  ergodic  sequence, 
Assume  that  the  conditions  of  theorem  1  are  true.  Then 


lim  Sn  =  Ee^,  a.s. 
n->» 


Proof.  We  have 
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r 


1  *1  .'iU'  iWl:  *  A  v  .La  J 


■1 


a  ,  n 

Sn  n  l 
i=l 

1  I? 

■s 

i=l 


=  J1  +  J2  +  J3. 


*  il5  ’  *1<Sn  ’  8-})‘ 


!  ^eixJ(!n 

i=l 


i)+l  l 
1=1 


In  view  of  ergodicity  of  the  sequence  [4],  we  gain 


lim  J  =  Eef,  a.s. 
n^°  J 


Write  M  =  suplJx. ||,  then  M  <  «.  Hence 
i  -1 

J2  <  M2|jen  -  6]!  0,  a.s. 

Finally,  lleix! (en  -  8)||  <  (|a2|  +  |a1l)M||  8n  -  e||,  so  we  obtain 


-  0,  a.s. 


By  (1 9)— (22) ,  (18)  follows. 


5.  STRONG  CONSISTENCY  OF  6n 

In  this  section  we  shall  prove  the  following  theorem: 

Theorem  5.  Suppose  that  each  e^  has  a  support  included  in  the  bound¬ 
ed  intervals  [a.,  82]  and  for  any  subsequence  {e^  :  j  =  1,  2,  . ..}  of  (e^) 

J 

we  have 


lim  e.  =  a0,  lim  e.  .  =  a-. ,  a.! 

.  1  i  C  .  1  i  * 

j-H»  j  J 


Also  suppose  that  {x.}  is  bounded,  and  for  any  nonzero  p-vector  d 


i  m  x*  d  ^  0 


lim  6  =  B,  a.s.  (24) 

%  *  *  <• 

Proof,  Without  losing  generality,  we  suppose  that  |a^J  <  a 2, 
othersise  consider  the  model  -yn  3  x'n(-B)  “  en« 

Several  preliminary  facts  are  in  order. 

(I)  Define  a  function 

g(a)  =  Tim  xjg 

on  Rp.  Then  g(a)  is  continuous  on  Rp,  and  g(a)  >  0  when  a  f  0. 

The  second  conclusion  follows  directly  from  the  assumption  of  the 
theorem,  and  the  first  is  an  easy  consequence  of  the  boundedness  of  (x^ 
Denote  by  B  the  surface  of  the  unit  sphere  in  Rp,  i.e. 

B  =  {a:  Io||*  1}.  Because  B  is  compact,  g  is  continuous,  and  g(o)  >  0 

for  a  f  0,  we  have 


c  =  inf  g(a)  >  0. 


(II)  Define  B  and  c  as  above,  then  we  can  find  a  positive  integer 
m  and  a  subset  "D  =  ,  .  ...g-}  of  B  such  that  for  any  a  e  B,  there 

exists  a..  €  T)  with  the  property 


sup{ | (a  - 


j  =  1 ,  2,  . . .}  <  c/3. 


(2> 


The  proof  has  been  given  in  §2. 

For  every  a.  €  E,  we  can  find  out  a  subsequence  {x^.}”_i  of  Ixn) 
such  that 


lim  a!x.  .  *  TTm  a!x„  4  s  i  2  m 

.  -i-i  i  -i«n»  '  1  •  c  »  •••»  m . 


Denote  the  values  of  y  and  e  corresponding  to  x. .  by  y..,  e..,  i .e 

•»  '  J  *  J  ■  J 

if  5ij  =  ?k-  the"  *1j  =  V  eij  =  ek-  Put 


F1  =  ™  *1  j  =  V 

m 

F  =  n  r  (2 

i=l  1 


(III)  P(F)  =  1. 

In  fact,  since  P(Fi )  =  1 ,  i  =1,  ....  m  by  (23),  we  get  the  result 
Employ  the  symbol  u  to  represent  a  real  number  sequence  ( c^ ,  c2,  . 
al  1  a2»  1  =  1*  2,  ...  .  The  event  F  determined  by  (27)  can  be 
viewed  as  a  subset  F  of  the  u-s pace,  i.e.  F  =  { ( ,  e2,  ...)  6  F> . 

(IV)  For  any  e>  0,  t  >  s,  0.  *  D,  put  d  =  8  -  tci\ .  Then  for  any 
a)  «  F,  there  exists  N.-(aJ’)  such  that 


max  |yk(w)  -  xj^dj  >  a  + 
l<k<n 


(2 


when  n  >  N.(<d). 
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Proof.  For  simplicity  of  notation,  we  shall  use  y^,  e^  to  express 
y^ (uT)  and  e^ol),  when 

4  '  ii3t  ■  ik£  +  ek  '  5k(S  -  ‘“i1  '  ek  +  ‘?k“i ■  (29> 

Take  n  =  mi'n{a,,  i .  Since  lim  e. .  =  a0  and  lim  alx..  = 

2  3(e+l)  u  2  j*. -1-10 

1  im  a!x  ,  we  can  find  k  =  k("n,tU)  such  that 
n-x*>  ~1~n 

eik  >  a2  "  (30) 

ojXik  >  c  -  n.  (31 ) 

Take  N.  =  N. (n,oi)  large  enough  so  that  when  n  >_  we  have 

~ik  €  {?1 •  **•*  ?n}* 

Then  by  (29 )-( 31 ) ,  when  n  >_  we  have 

max  |yk  -  x'd  |  >  |e,k  a  tx,g,| 
l<k<n 

>  -  n  +  t(c  -  n)  3  *2  +  ^  +  ^)n 

2  — 

^  a2  +  -jtc 

since  TTFiTT  -  W+TJ  ,orti  e- 

This  proves  the  assertion. 

Now  turn  to  the  proof  of  Theorem  5.  Fix  w  €  F  and  e  >  0,  and  again 

use  yi ,  e.. ,  N.. ,  etc.  to  express  y^  (w) ,  e^  (ul) ,  N.(to).  Put  N  =  N(u>)  = 

max  N.(u>).  Suppose  that  l  6  Rp  satisfies  ||£  -  s]j  >  e,  then  l  can  be 
l<i<n  '  “  “ 

written  as  l  -  B  -  ta,  t  >  e,  <*  €  B .  By  (II)  we  can  choose  a.  6  D  so 

that  (26)  is  true.  Remember  that  d^  =  §  -  to^  ,  we  have 
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h  -  ?iif '  yk  •  S(5  •  UJ  =  yk  •  fkc?t +  t(?i  - ;» 

'  yk  "  Jk2t  '  l!!k*5  '  ?)'• 

Then 

|yk  "  -  lyk  "  ii&J  "  1*/3* 

From  this  inequality  and  (28),  we  have 

max  [y.  -  x!l\  >  max  | y.  -  x/dj  -  tc/3 
l<k<n  K  ~K~  ~  l<k<n  K 

>  a^  +  ftc  -  tc/3  =82+  ^tc 

for  n  >_  N.  On  the  other  hand,  we  know  that 

max  |y.  -  x/8|  =  max  je.  |  £  a?. 

1  <k<n  l<k_<n  K  c 

So  we  have  Qn(£)  >  Qn(s)  for  n  ■>  M  and  ||£  -  b||  _>  e.  According  to  the 
definitions  of  B-  ,  we  have  | Fn  -  6 1{  <  e  when  n  N.  Since  e  >  0  is 
arbitrary,  this  proves  that  lim?n  =  B  when  u  e  F.  The  proof  of  Theorem  5 
is  completed  in  view  of  (III).  n 


6.  MEAN  SQUARE  CONSISTENCY  OF  Bn 
In  this  section  we  shall  prove  the  following  theorem: 


Theorem  6.  Under  the  assumption  of  theorem  5,  we  have 

lim  Efflf  -  §||2)  =  0. 

n-wo 

Proof.  Without  losing  generality,  we  suppose  that  |a^|  <  a ^ • 

Define  h ( oc )  =  lim  max  a'x..  Because  (x. }  is  bounded,  "F(a)  is 
n~*»  1 <i<n~  ~ 

finite  everywhere  and  further  it  is  easily  seen  from  the  assumption 
of  theorem  5  that  h(a)  >  0  when  f  0,  and  K(a)  is  continuous  everywhere 

on  Rp.  So  the  infimum  of  h(a)  on  the  surface  of  unit  sphere  B  is 
greater  than  zero:  s  =  inflT^a)  =  a  «  B}  >  0.  Find  on  B  a  finite  sub¬ 
set  6  =  (B-j,  ...»  6^1  with  the  following  property:  For  any  a  €  B,  there 
exists  i  €  {1,  ...,  r>  such  that 

sup{  |  (a  -  6  .• ) '  x  ■  |  :  j  =  1,  2,  ...}  <  s/4.  (32) 

-  >  -  J 

Now  we  prove  the  existence  of  N  such  that 

max  x!a  >  s/4 

for  every  a  €  B  when  n  _>  N. 

In  fact,  by  the  definition  of  s,  we  can  find  out  N  such  that 
max  x'a.  >  3s/4,  i  =  1 ,  ....  r 

llKl" 

when  n  _>  N.  For  any  a  e  B  choose  a.  such  that  (32)  holds.  Then  we  have 

max  x'a  =  max  (x/a.  +  x'(a  -  a.)) 
l<k<n~K~  l<k<n  ~  *’1 

>  max  x£a .  -  s/4  >  s/2 
l<k<n'K”1 

when  n  _>  N.  Now  put  t  *  ^2^-  If  l  €  Rp  and  ||£  -  s||  >  t,  then  l  can 


be  written  as  i  =  B  -  ta,  a  e  B,  t  >  t.  So  that  when  n  _>  N,  we  get 

Q_U)  =  max  | y,  -  x/£|  =  max  |e.  +  tx'aj 
l<k<n  ~  ~  1  <k<n 

_>  t  max  |  x/aj  -  max  je.  j 
l£k_<n  ~  ~  1  <k<n 

>  ts/2  -  a2  >  a2  -  0 


This  shows  that  (S^,  3N+-| ,  ...)  are  uniformly  bounded.  Since  P(lim 

n-*» 

=  1  by  theorem  5,  the  assertion  of  theorem  6  follows  from  the  dominant 
convergent  theorem. 


|0Q  | 
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7.  ESTIMATION  OF  THE  ENDPOINT  OF  THE  SUPPORT  WHICH 
HAS  THE  MAXIMUM  ABSOLUTE  VALUE  AND  SECOND  MOMENT 

Define  ap  =  max  |y.  -  xjj  |  . 
l<i<n 

A 

Theorem  7.  Under  the  assumption  of  theorem  5,  a^  is  a  strong  con¬ 
sistent  estimate  of  max  { I a^ !  , 

Proof,  Without  losing  generality,  we  suppose  that  | a-j  J  <  a^. 

Because  lim  max  e.  _>  lim  e  ,  by  (23)  we  have  P(1im  max  e.  =  a„)  =  1. 
n-w  1_<i_<n  n  n-*»  l_<i  <n  " 

Now 

max  |y .  -  x!s  |  =  max  |y.  -  xjB  +  x!(e  -  J  )| 

1 < i <n  “  1<i<n 


=  max  |y.  -  x! B |  +0  =  max  e.  +  J 

1<i<n  1  n  l<i<n  1  n 

where  |J  [  £  supl  X-j  II  ||Jn  -  B|| .  Because  Tim  Sn  =  B ,  a.s.  and  { x .  > 

is  bounded,  we  derive  that  lim  Jn  =  0,  a.s.  Combining  this  with 

n-x» 

A 

P(lim  max  e.  =  a„)  =  1 ,  we  get  lim  a  =  a~,  a.s. 
n-+°°  1<i^n  n-H» 

Now  suppose  that  e-j,  e2»  ...  is  a  strictly  stationary  ergodic 

2  _  i  _  2 

sequence.  As  an  estimate  of  Eef ,  we  use  s =  —  Y  (y.  -  x'.8n)  • 

I  n  n  *  i  **  •  «*n 

Theorem  8.  Suppose  that  (e - }  is  a  strictly  stationary  ergodic  se¬ 
quence.  Assume  that  the  conditions  of  Theorem  5  are  true.  Then 

_  2 

lim  s  =  Ee, ,  a.s. 
n  i 

A 

If  we  substitute  Bn  by  Bn»  the  proof  is  the  same  as  the  proof  of  Theorem  4. 
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